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Abstract. It is shown that the deformed Calogero-Moser-Sutherland (CMS) 
operators can be described as the restrictions on certain afhnc subvarieties (called 
generalised discriminants) of the usual CMS operators for infinite number of parti- 
cles. The ideals of these varieties are shown to be generated by the Jack symmetric 
functions related to the Young diagrams with special geometry. A general structure 
of the ideals which arc invariant under the action of the quantum CMS integrals is 
discussed in this context. The shifted super- Jack polynomials are introduced and 
combinatorial formulas for them and for super- Jack polynomials are given. 

1. Introduction 

The primary goal of this paper is to explain the algebraic nature of inte- 
grability of the deformed Calogero-Moser-Sutherland (CMS) operators 



+ > -^97 T+> > (1) 



sin2(xi -Xj) ' ^.sm^ivi-yj) ' ^ ^ sin^(xi - y^-) 

For m = 1 these operators have been first introduced in P3I2], for general m 
they were considered in In [S] we have suggested a general construction 

of the deformed CMS operators related to Lie super algebras, which in the 
case of Lie superalgebra sl{n\m) leads to the operators (0). Unfortunately 
this relation with Lie superalgebras itself does not supply the integrability of 
this problem, which was proved in 5 by direct construction of the quantum 
integrals. 

In this paper we will present a more conceptual proof of the integrability 
of by showing that (after some gauge transformation and change of 
variables) the deformed CMS operators can be described as the restriction 



of the usual CMS operators for infinite number of particles onto certain 
subvarieties of Macdonald variety called generalised discriminants. Note 
that the restriction of a differential operator onto a submanifold is possible 
only under very special circumstances. In case of the algebraic subvariety 
this means that the corresponding ideal must be invariant under the action 
of the operator. 

For the proof we use the theory of Jack polynomials , and the theory 
of shifted Jack polynomials developed recently by Knop, Sahi, Okounkov 
and Olshanski El • We have been partially inspired by a recent very 
interesting paper by B. Feigin, Jimbo, Miwa and Mukhin, where certain 
ideals in the rings of symmetric polynomials were described in terms of Jack 
polynomials. 

The structure of the paper is following. First we review the basic facts 
from the theory of Jack and shifted Jack polynomials and from the theory 
of Cherednik-Dunkl operators. 

In section 5 we introduce the generalised discriminants and prove our main 
result. In section 6 we show that the quantum integrals we have constructed 
in [2] can also be described as the restrictions of certain integrals of the usual 
CMS problem. The notion of the shifted super-Jack polynomials naturally 
appears in this relation. 

Section 7 is devoted to the description of the ideals in the algebra of 
symmetric functions which are invariant under the action of the quantum 
integrals of the CMS system. We show that the rectangular Young diagrams 
related to generalised discriminants play a very special role here. 

In the last section we give combinatorial formulas for the super-Jack and 
shifted super- Jack polynomials generalising the results of Stanley, Okounkov 
and Olshanski [311131121. 

2. Symmetric functions and Jack polynomials 

In this section we recall some general facts about symmetric functions 
and Jack polynomials mainly following Macdonald's book It would be 
convenient for us to use instead of the parameter a in Macdonald's notations 
of Jack polynomials the parameter 




(cf. jlOjl. It is different from the parameter k used in our previous work [31 
by sign change: 

d = -k. 

Let P/v = C[xi, . . . , xat] be the polynomial algebra in independent 
variables and Atv C Pn be the subalgebra of symmetric polynomials. 
A partition is any sequence 



A — (Ai, A2, . . . , Ar- . . . ) 
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of nonnegative integers in decreasing order 

Ai > A2 > • • • > > . . . 

containing only finitely many nonzero terms. The number of nonzero terms 
in A is the length of A denoted by 1{X). The sum | A |= Ai + A2 + • • • is called 
the weight of A. The set of all partitions of weight is denoted by Vn- 

On this set there is a natural involution: in the standard diagrammatic 
representation it corresponds to the transposition (reflection in the main 
diagonal). The image of a partition A under this involution is called the 
conjugate of A and denoted by A'. This involution will play an essential role 
in our paper. 

Partitions can be used to label the bases in the symmetric algebra Atv- 
There are several important bases in A^v- 

1) Monomial symmetric polynomials m\,\ € Vn are defined by 

mx{xi, . . .,xn) = '^xl^x'^^ ■■■x'^N 

summed over all distinct permutations a of A = (Ai, A2, • • • , Xn)- 

2) Elementary symmetric polynomials are defined by 

00 

^efct'= = JJ(l + Xit). 

k=o i>l 

For each partition A we define 

e\ = CAiCAa • • • 

3) Similarly complete symmetric polynomials are defined by 

00 

^hkt'^ = Y[{i-x,tr' 

k=o i>l 

and 

h\ = hx-^h\2 

4) Finally the most convenient for us will be the power sums 

Pk = x'l + x^ + 
where again for any partition A 

PX = PX1PX2 ■ ■ ■ 

It is well-known [2] that each of these sets of functions with < N 
form a basis in Atv- 

We will need the following infinite dimensional versions of both P/v and 
Atv. Let M < N and ipN,M '■ Pn — > Pm be the homomorphism which 
sends each of xm+i, ■ ■ ■ ,xn to zero and other Xi to themselves. It is clear 
that (pN,Mi-^N) = Aa/ so we can consider the inverse limits in the category 
of graded algebras 

P = limPM, A = limAAr. 



This means that 

A = e^oA^ A" = limA:v 

where PJ^, are the homogeneous components of Pj^, A^r of degree r. The 
elements of A are called symmetric functions. 
Since for any partition A 

'/'Af,M("iA(a;i, . . .,xn)) = mx{xi, . . .,xm) 

(and similarly for the polynomials h, e,p) we can define the symmetric func- 
tions mx,hx,ex,px. 

Another important example of symmetric functions are Jack polynomials. 
We give here their definition in the form most suitable for us. 

Recall that on the set of partitions there is the following dominance 
partial ordering: we write < A if for alH > 1 

/"I + H + Aij < Ai + A2 H 1- Aj. 

Consider the following CMS operator 

i=l ^ *^ i<«<i<A^ « J \ * J/ « 

1=1 ■' 

It coincides with the standard (trigonometric) Calogero-Moser-Sutherland 
operator ^1 if we change the gauge and use exponential coordinates. It 
is related to the so-called Laplace-Beltrami operator 

i=l V i^j ' J * « 

used by Macdonald [HI by a simple formula 

N 

. , dx, 

1=1 

An important property of the CMS operator is its stability under the 
change of N : the following diagram is commutative 

A^ ^ A^ 

^(M) 

Am — > Am 

(see [0], example 3 on the page 326). This allows us to define the CMS 
operator Cq on the space of symmetric functions A as the limit of 



Theorem-Definition. If is not negative rational number or zero than 
for any partition A , 1{X) < N there is a unique polynomial Px{x,9) € An 
(called Jack polynomial) such that 

1) Pxix, 9) = mx + J2f,<x where G C 

2) Px{x,d) is an eigenfunction of the CMS operator jC,g^\ 

To prove this one can check that the operator JC-^J^^ has an upper triangular 
matrix in the monomial basis m^: 

cf\mx) = Y^Cx,m, 

where the coefficients c\^ can be described explicitly (see [£], page 327 and 
jl5j). In particular 

N N 

cxx = ^>^1 - 26^ ^{i - l)Xi = 2n{X') - 2dn{X) + |A| 

i=l 1=1 

where n{X) = ^j>i(i — l)Ai. It is easy to see that if A > then n(A) < n(/x) 
and n(A') > n(fi'). So we have 

CAA - c^f. = 2[n{X) - n{fi)] + 2e[n{fi') - n{X')] 

Since 9 is not a negative rational we see that caa 7^ c^^ if ^ > This 
implies the claim. 

From the stability of the CMS operators it follows that 

(PN,M{P\ixi, . . .,Xn)) = Px{xi,. . . ,Xm) 

SO we have correctly defined Jack symmetric functions P\{x,9) € A which 
are the eigenfunctions of the CMS operator Cg. 

3. Shifted symmetric functions and shifted Jack polynomials. 

We discuss now the so-called shifted Jack polynomials introduced recently 
by Knop, Sahi, Okounkov and Olshanski (8. .9. .10. il2) . For a nice review of 
the theory of shifted symmetric functions and its role in the representation 
theory of the symmetric group we refer to ^H] • 

Let us denote by Ajy^g the algebra of polynomials /(xi, . . . , x^v) which 
are symmetric in the shifted variables Xi + 9(1 — i). This algebra has the 
filtration by the degree of polynomials: 

{AN,e)o C (A7v,e)i C • • • C (A7v,e)r C. . . 
We have the following shifted analog of power sums: 

N 

Plixi, ...,XN,9) = Y^ [{x, + 9{l - i)Y - {9{1 - i)Y] . (3) 

i=l 

The polynomials 

pI{x,9)=pI^{x,9)pI^{x,9)... 
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where A = (Ai, A2, . . . , A^, . . . ) is a partition of N form a basis in A]y^0. Since 
^N,M{AN,e) = ^M,e one can consider the inverse Umit 

in the category of filtered algebras: 

00 

= y {Ke)r, (Ae)r = lini(Aiv,e)r. 

r=0 

The algebra Kq is called the algebra of shifted symmetric functions. 

The shifted Jack polynomials can be defined in the following way (see 
[2j,^n])- Let us introduce the following function on the set of partitions 

H{\e) = \{{cs{n) + i) (4) 

□eA 

Here we use the diagrammatic representation of the partitions using the 
squares [H] and to each square □ = {i,j) we prescribe the numbers 

c^{u) = \-j + e{>:^-i). (5) 

Theorem-Definition jnHTOJ. Let X he a partition with X^+i = 0. There 
exists a unique shifted symmetric polynomial P^{x,9) £ Atv.s (called shifted 
Jack polynomial) such that degPx < |A| and 

H{X), f, = X 

0, < |A|,/x ^ X,fiN+i = 



Here and later throughout the paper by P{X) for a polynomial P{xi, . . . ,xiy) 
and a partition A = (Ai, . . . , Xjy) we mean P(Ai, . . . , Xn)- It is actually a 
very useful idea to view the shifted symmetric polynomials as the functions 
on the partitions (see e.g. [T6j). 

It is easy to see from the definition that the shifted Jack polynomials are 
stable: 

^PN,MiPx{xi, ■ ■ .,XN,0)) = Pxixi, . . .,XM,9), 

SO we can define the shifted Jack function P^(x,0) as an element of Aq. 

Knop and Sahi ^ proved that the shifted Jack polynomials satisfy the 
following so-called Extra Vanishing Property: 

PIM = Q (6) 

unless the diagram of A is a subset of the diagram of /x and that P\{x, 9) is 
the usual Jack polynomial Px{x,9) plus lower order terms. 

We will need the following duality property of the shifted Jack polynomi- 
als: 

For shifted Macdonald polynomials this was proved by Okounkov jl^- We 
present here an independent proof for Jack polynomials based on the Bernoulli 
sums. 



Consider the following natural conjugation homomorphism: 

{u:*{mX) = f{X') (8) 

Proposition 1. The conjugation homomorphism maps the algebra of 
shifted symmetric functions Ag into the algebra A.i/g- 
Proof. Let us introduce the following Bernoulli sums: 

N 

bk{xi,...,XN,0) = Y,[Bk{xi + e{l-i))-Bk{9{l-i))], (9) 

i=l 

where Bk{x) are the classical Bernoulli polynomials. Recall that Bk{x) can 
be defined through the generating function 



- = J2Bk{x) 

and satisfy the property 



k=0 



Bk{x + I) - Bk{x) = kx^-^ 

or, more generally 

I 

Bk{x + l)-Bk{x) = k^{x + i-lf-^. (10) 
It is easy to see that the Bernoulli sums are stable: 

VN,M{bk{xi, . . .,XN,0)) = bk{xi, . . .,XM,6), 

so one can define the shifted symmetric Bernoulli functions bk{x,6) G Ag. 
Lemma 1. Bernoulli functions satisfy the following symmetry: 

bk{x',e) = {-e)^-%{x,i/e). (ii) 

The proof is a straightforward check: 

bu{x\e) = [Bk{x^j + e{i - j)) - Bu{e{i - j))] = 
j=i i=i 

k m -J)+^- = K-ef-^ [ 1]'"' 

i=l i=l 

Y [^^-(^^ + - ^)) - ^^(^(1 - ^))] = i-(^)'''hiX, 1/9) 

i=l 

We have used here the property (|10|) of the Bernoulli polynomials. 



Since Bernoulli sums bk{x, 9) generate the algebra Ag the proposition now 
follows. Combining this with the definition of the shifted Jack polynomials 
we have the duality property 

4. ChEREDNIK - DUNKL OPERATORS AND HARISH-ChANDRA 

HOMOMORPHISM 

In this section we present the basic facts about Cherednik-Dunkl opera- 
tors. For the details we refer to Opdam's review |18j . 

By Cherednik - Dunkl operators we mean the following difference-differential 
operators 

A,^ = x.^+e5]^^^^^(l-a.,), ^,j<N, (12) 

OXi , Xi Xj 

where CTij is acting on the function ^(xi, . . .xm) by permutation of z-th and 
j-th coordinates. If i > we assume that -Dj^at = 0. We should warn the 
reader that our definition is different from the usual one 

UXi , X 2 Xj 

(see e.g. |18| ') by a shift. This shift is necessary for the stability of D^^jq and 
explains the relations with the theory of shifted symmetric functions which 
we discuss below. 

The first important property of the Cherednik - Dunkl operators is that 
they commute with each other: 

[A,7v,^i,jv] = 0. 

This means that one can substitute them in any polynomial P in variables 
without ordering problems. 

The second property is that if one does this for a shifted symmetric poly- 
nomial / G Ajv^e then the corresponding operator f{Di^N . . . D]y^]\f) leaves 
the algebra of symmetric polynomials Ajv invariant: 

/{Di^N ■ ■ ■ Dn,n) ■■ Atv ^ Atv. 

The restriction of the operator /(-Di,Ar . . . -Dat.at) on the algebra A^v is given 
by some differential operator, which we will denote as A formula 

for this operator can be found by moving all the permutation operators 
in /(Di^Tv • • • -Djv^Tv) to the right using natural commutation relations and 
then erasing them. 

One can check that if we apply this operation to the shifted square sum 
p*2{xi,...,XN,0) = E£i + ^(1-0)^- (^(1-i))^] we arrive at the 
CMS operator ((2j. Thus all the operators >Cy^g are actually the quantum 
integrals of the CMS problem. The Jack polynomials are the joint eigenfunc- 
tions of all these operators: if P\{x, 9) is the Jack polynomial corresponding 
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to a partition A of length N then 

£^j^,Px{x,e) = f{Xi,X2,. . . ,XN)Px{x,e.) (13) 

This allows us to define a homomorphism (which is actually a monomor- 
phism) X '■ f ~^ ^Ne from the algebra An^q to the algebra of differential 
operators. Let us denote by ^{N, 6) the image of x- The inverse homomor- 
phism 

X-^ -V^N^e)^ KN,e 
is called the Harish- Chandra isomorphism. It can be defined by the action 
on the Jack polynomials: the image of £ G 'D^N, 9) is a polynomial / = 
fc S Ajv^e such that 

CPx{x,9) = f{X)Px{x,e). 
One can check that the Cherednik-Dunkl operators Di^n are stable (note 
that this is not true for the shifted operators Di^i\f): the diagram 

Pn ^ Pn 
Pm — > Pm 

is commutative for all M < and i > 1. Similarly for any / E A^r^e and 
g = (pN,M{f), M < N the following diagram is commutative: 

An — > An 

i 'PN,M i ^N,M 

Am — > Am 

This allows us to define for any shifted symmetric function / € a differ- 
ential operator 

Cl:A — .A 

and the infinite dimensional version of the homomorphism x- We will de- 
note by T^iO) the image of this homomorphism, which can be interpreted as 
the ring of quantum integrals of the (infinite-dimensional) CMS problem. 
The inverse (Harish-Chandra) homomorphism x^^ ■ ^(^) — ^ Ag can be 
described by the relation 

4p{x,e) = f{x)p{x,9), 

where now f € Ag and P{X, 9) is Jack symmetric function. 

5. Generalised discriminants and deformed CMS operators. 

Now we are ready to formulate our main results. The central role in our 
construction will play the following algebra A„ ^.e introduced in 5 (denoted 
there as Al^^ ^). 

Let Pn. ra — C[xi, . . . , Xn: Ui: ■ ■ ■ : ym] be the polynomial algebra in n -\- m 
independent variables. Then An^m,e C Pn,m is the subalgebra consisting 

9 



of polynomials which are symmetric in xi, . . . , Xn and yi, . . . , separately 
and satisfy the conditions 



or, equivalently 



on each hyperplane Xj — yj = for i = 1, . . . , n and j = 1, . . . , m. It is 
shown in [S] that for generic 6 (namely, if 9 is not a negative rational or 
zero) An^mfi coincides with its subalgebra Mn,m,e generated by the deformed 
Newton sums 



n ^ m 

E<-^Eyj (16) 



Pr[x,y,ti) = 

i=l - j=l 



which obviously belong to An,m,e for all nonnegative integers r. 

This algebra has appeared in [Hj in relation with the following deformed 
CMS operator 



1=1 ^ ' j=i ^ ■' ' i<*<j<'i 

i<t^<m y^ ~ yj ^ ^y^ ^y^ ^ " y^ ^ ^y^ 



1=1 j=l ■> J 

Lemma 2. The deformed CMS operator preserves the algebra An,m,e'- 

The proof follows from a more general statement proved in the next sec- 
tion (see Theorem 5). 

Let now A be the algebra of symmetric functions in infinite number of 
variables zi, Z2, ■ ■ ■ , Pr{z) = + Z2 + ■ ■ ■ be the power sums, Px{z, 9) be the 
Jack polynomials (see Section 2 above). 

Consider the following homomorphism ip from A to A,„ „j g such, that 

^{Pr{z)) =Pr{x,y,9). 

Since Priz) are free generators of A this determines ip uniquely. Such a 
homomorphism was first used by Kerov, Okounkov and Olshanski in |19j . 

Our central result can be formulated as follows. Let Cg be the usual CMS 
operator in infinite dimension. 
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Theorem 1. The following diagram is commutative for all values of the 
parameter 9: 

A 

i ^ (19) 

Before going to the proof let us discuss the geometric meaning of this 
claim. For generic values of the parameter 6 according to [3] the algebra 
^n,m,e is finitely generated, so we can introduce an affine algebraic variety 

T^n,m,e = SpecAn,m,e = SpecJ\fn,m,e- 

We call this variety generalised discriminant because for special values m = 1 
and 9 = —1/2 the algebra Nn^i,-i/2 coincides with the algebra of functions 
on the standard discriminant variety T^n+i of the polynomials of degree n + 1 
having a multiple root. A more general case 9 = — 1/g corresponds to the 
stratum in the discriminant variety when one of the roots has a multiplicity 
at least q, so informally speaking the generalised discriminant Vn^mfi consists 
of "polynomials" with n simple roots and m roots of "multiplicity" /i = 
-9-\ 

We understand that our terminology is not perfect since the term "dis- 
criminant" was used and generalised in many different ways (see e.g. well- 
known book [20] by Gelfand, Kapranov and Zelevinsky) but an alternative 
term "generalised coincident root loci" looks too long and not much better. 
We would like to mention that the problem of finding the algebraic equa- 
tions defining the strata in the discriminants is non-trivial and goes back to 
Arthur Cayley [U (see [221 1231 124j for the recent results in this direction) . 

The map (p determines an embedding of T>n,m,e into M. = Spec A. We will 
call M Macdonald variety although strictly speaking it is defined only as an 
affine scheme since the algebra A is not finitely generated. Since the algebra 
^n,m,e is also not finitely generated for special positive rational 9 (see _5 ) 
the Macdonald variety is a proper space for the generalised discriminants to 
live in. 

Notice that for negative rationals the algebra A„ „ 6» could be bigger than 
m,9j SO in general we should distinguish the variety 'Dn,m,8 — Spec Kn^mfi 
and its embedding in M. Vn.mfi = SpecMn,m,e- 

Corollary. The deformed CMS operator JiTj ) is the restriction of the 
usual CMS operator Cq on Macdonald variety onto the generalised discrim- 
inant suhvariety T>n.mfi- 

We should mention that the restriction of a differential operator C onto a 
subvariety is a very rare phenomenon. Indeed this is possible only when the 
ideal corresponding to this subvariety is jC-invariant. In our case the situa- 
tion is even more peculiar because we have a finite-dimensional subvariety 
in infinite dimensions. 
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Proof of Theorem 1. Let us introduce the following function 11 G 
A[[ti, . . . ,tN]] which plays an important role in the theory of Jack polyno- 
mials (see 13): 

N 

1=1 i>l 

Lemma 3. The function H satisfies the following properties: 
(i) 

£e,z^ = 4^)n, (20) 

where index z (resp. t) indicates the action of the CMS operator Cq 
on z (resp. t) variables 

(ii) 

N n m 

^(n) = n 11(1 - x.ti)-' Hil- y,ti) (21) 
1=1 1=1 j=i 

(iii) 

f{Ce,zm = jC-n,mM'n) (22) 
Proof. Introduce the notation cu = The following identities are 

easy to verify 

ZjCii - ZiCji tidk - tkCii d . 9 . , . 

= CilCji, = CiiCik, Zi — [Cii) = ti — [cu) = Cu[Cu + l). 

Zi — Zj tk — zi ozi oil 

Therefore we have 

u-^zi— in) = e^cii; u-Hi—(u) = e^cu, 
^~'(^'^) (n) = ^' (^E^^') +0^Qi(Qi + i), 



Therefore 

n-'CeA^) = E (E ^^') + ^ E ^^'(^^^ + 1) + 2^' E E ^^'^i' 

i \ I / i,l i<j I 



and 



n-i/:g) (n) = E (E ^*') + ^ E (^^' + 1) + 2^' E E 

I \ i / Li k<l i 



which are equal. This proves the first formula 

To prove (|21j) let us first note that since 93 is a homomorphism it is enough 
to consider the case = 1 when we have only one variable t. Introduce now 
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the following automorphism uq which is defined by his action on the power 
sums as follows: 

(^eiPriy)) = —gPriy)- (23) 

Then we have 

^ [^{l - Zit)-'^ = - Xitr'ae (^{l - yjty 

Y[{1 - Xity'ae |^explog]J(l - = 



i>i ^ J i \ i>i 

n m 

Hii-xity'Hii-yjt). 

i 3 

This proves the second part of Lemma 3. 

To prove the last part introduce an = j^^^^,bji = i^Iy.^ ■ We have 

ifiU)-^ (x~^ ^(n) = ^ ai}j + e ^ au{au + 1) j , 

Now taking into account the following identity 

Xi yj 

we can write 

n / \^ / ™/ \^ 



i=l \ I / \ i,l J=l \ I ' 3,1 

13 



-6'^'^'^{aii + bji + 2aiibji) + 6'^ ^ '^{au + aji + 2auaji) 

i=l j=l I l<i<j<n I 



+ E ^ibil+bjl + 2bab,l)-{9{n-l)-m)ieY,a^l-J2bJl 
and 

« \ i i I i,l i,/ 



e ^ OizOifc - ^ bjibjk ■ 
i=l j=l J 

It is easy to check that the last two expressions are identical, so (pCH.)'^ Cn,m,ev(^) = 

(^(n)-i4^V(n) = (^(n)-V(4!I^(n)) = v^(n)-V(/:e,.(n)). This com- 

pletes the proof of Lemma 3. 

To complete the proof of Theorem 1 we should show that the coefficients 
g\{z,0) in the expansion of the function 

n = - Zitj)-' = ^gx{z,0)mx{t) 

generate A when we increase the number of variables t. But this follows from 
the relation (see [H], 1.4) 

lll(^-z^t,)-A = llil-z,t,) = J2i-^)^^^ex{z)m,{t), 

\ i,j J iJ A 

where ex and mx are the standard symmetric functions defined in section 2, 
and the fact that ex form a basis in A. Theorem 1 is proved. 

We believe that a similar statement true for any quantum integral jCf € 
T>{9) of the CMS problem, which is equivalent to the fact that the kernel 
of if is generated by certain Jack polynomials. For generic values of the 
parameter 9 this follows from the following Theorem 2. 

Let us introduce the set of partitions Hn^m-, which consists of the partitions 
A such that \n+i < rn or, in other words, whose diagrams are contained in 
the fat {n,m) - hook (see fig.l). Its complement we will denote as Hn,m- It 
consists of the diagrams which contain the (n + 1) x (m + 1) rectangle. 
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Fig. 1 : Fat (n,m)-hook 
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Theorem 2. If 6 is not a negative rational number or zero, then Kenp 
is spanned by the Jack polynomials P\{z,9) corresponding to the partitions 
which are not contained in the fat {n,m)-hook (or equivalently, which contain 
(n + 1) X (m + 1) rectangle). 

Proof. Notice first of all that if is not a negative rational number or 
zero the Jack polynomials Px{z, 9) are well defined, otherwise in general they 
may not exist. 

Let us consider the following automorphism (see [7], [S], VI. 10) ujq of 
algebra A : 

where pr are the standard power sums. Then according to [H] (see page 380, 
formula (10.17)) we have 

where as before A' is a partition conjugate to A. 

Let now x = (xi, X2, . . . ), y = {yi,y2, ■ ■ ■ ,) be two infinite sequences vari- 
ables. Then we have (see page 345, formula (7.9 )) 

Px{x,y,e) = Y,Px/^.{x,e)Pf,{y,e) (24) 

where Pxi^{z,6) are the skew Jack functions defined in jH] (see Chapter 6, 
§ 7, 10) and n C X means that /Xj < Aj (or equivalently the diagram of is 
a subset of the diagram of A). 

Now consider the automorphism ae which is combination of the automor- 
phism uji/g with the change of sign yi — > —yi. The action of ag on the power 
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sums is given by the formula 

If we apply this automorphism acting in y variables on both sides of the 
formula (|24)) and put all the variables x and y except the first n and m of 
them to zero we get 

^{P,iz,9)) = E(-l)'"'^A/,(x,^)^^;|^^^P,'(?/,^-^)- (25) 

fiCX \P I ) 

Now let us assume that A is not contained in the fat (n, m)-hook, then 
^m+i > We have two possibilities: l^-'m+i > or = 0. In the 

first case we have P^{yi, . . . , ym, 0~'^) = while in the second case X'^+i ~ 
f^'m+i > ''^j so according to 6 (page 347, formula (7.15)) the skew function 
Px/f^{xi, . . . ,Xn,0) = 0. Thus we have shown that the Jack polynomials 
Px{z, 6) with A S Hn,m belong to the kernel of ^p. 

To prove that they actually generate the kernel consider the image of the 
Jack polynomials P\{z,9) with A G Hn^m- From the formula (|25|) it follows 
that the leading term in lexicographic order of ip{P\{z,0)) has a form 

<A'i-n> „,<A',„-n> 



where A' = (A'l, A'2, . . . ) is the partition conjugate to A and < a >= ^^ilM = 
max(0, a). From the definition of if it follows that ip{P\{z,9)) G ^n,m,9- It 
is clear that all these polynomials corresponding to the diagrams contained 
in the fat hook are linearly independent. Theorem 2 is proved. 

Notice that we have also shown that for the generic 9 the polynomials 

SPx{x,y,9) = ip{Px{z,9)), X e Hn,m (26) 

form a linear basis in A„ ,„ These polynomials are called super-Jack poly- 
nomials (see j25j). 

Remark 1 . A natural question is what are the algebraic generators of the 
algebra An^m,e- One can show using Theorem 2 that for generic 9 the first 
mn + m + n deformed Newton sums ((TB|) pi{x^ y,9), . . . ,pmn+m+n{x, y, 9) 
(or equivalently the super-Jack polynomials SP\{x,y,9) corresponding to 
the Young diagrams with one row of length less or equal than mn + m + n) 
generate this algebra. The number mn + m + n here is the area of the 
rectangle (n + 1) x (m + 1) minus 1. One can use Theorem 2 to produce 
the relations between the generators (which are the equations determining 
the generalised discriminants) but a satisfactory description of them is still 
unknown. The fact that An^m,8 for generic 9 is finitely generated was proved 
earlier in 5 . 

Remark 2. For special 9 of the form — | the homomorphism ip can 
be passed through the finite dimenion N = n -\- mq: (p = (p o (^^, where 
ipN : A ^ A^r is the standard map (all Zi except go to zero), and : Aat — > 
An. m g is a homomorphism corresponding to identification of all zi except 
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n into m g-tuples. The variety Vri,m,e = SpecMn,m,e in this case can be 
interpreted as the stratum in the discriminant (known also as coincident root 
locus) consisting of the polynomials of degree A'^ with all but n roots having 
the same multiplicity q. In particular, if m = 1 we have the polynomials 
with a multiple root of multiplicity at least q. The corresponding ideal (the 
kernel of (j)) in this particular case was investigated by B. Feigin, Jimbo, 
Miwa and Mukhin in They showed that it is also spanned by the 

Jack polynomials but the geometry of the corresponding Young diagrams 
is much more complicated. This important paper shows that the case of 
special values of 6 is actually very interesting and deserves more investigation 
(see j24j for the latest development in this direction). As we have already 
mentioned to describe the algebraic equations of the discriminant strata is 
a classical problem which is still largely open 1221 123j . 

Corollary. Let f ^ Ag be a shifted symmetric function and Cg € be 
the corresponding quantum integral of the CMS problem. Then for generic 
9 there exists a quantum integral cI^^q of the deformed CMS problem Ji7| ) 
such that the following diagram is commutative 

A ^ A 

An,m.O . 

The super- Jack polynomials h2b}) are the joint eigenf unctions of all the op- 
erators jOI ,^ n. 

In the next section we investigate the homomorphism: / — > -C^ ^ g in 
more detail. 

6. Shifted symmetric functions and quantum integrals of the 

deformed cms problem 

Let again Pn,m — C[x-i, . . . , Xn, yi, . . . , ym] polynomial algebra in ti + ttj 
independent variables. The following algebra aJ^ ^ introduced in j^I can 
be considered as a shifted version of the algebra A„ g. It consists of the 
polynomials p(xi, . . . , Xn, yi, ■ ■ ■ , ym) which are symmetric in xi, . . . , Xn and 
yii - ■ ■ lUm separately and satisfy the conditions 

f{xi + 1/2, y,- - 1/2) = f{xi - 1/2, + 1/2) 

on each hyperplane Xi + Oyj = for i = 1, . . . , n and j = 1, . . . , m. It is easy 
to check that the deformed Bernoulli sums 

n m 

bUx, y,e) = J2 Br{x^ + 1/2) + {-0)"^-' Yl ^-(^i + 1/2) (27) 

i=l j=l 

belong to A^nme integers r > 0. According to ^ for generic 9 

b*{x,y,9) generate the algebra aJ^^^. 
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Now we are going to define the homomorphism which is a shifted 
version of the homomorphism 95 from the previous section. 

RecaU that lin,m denote the set of partitions A whose diagrams are con- 
tained in the fat (n, m)-hook. Consider the following Frobenius map F : 
Hn,m — ^ C"+'" : F(A) = {pi,...,pn,qi,..., Qm), where 



Pi = Aj 



1 

'-2 



1 

— {m 
2 



{m — 9n) , qj = fij 



3-1 



2 +2 



m + nj , 
(28) 

and /X = (A„+i, An+2) • • • )• The motivation for this particular shift comes 
from the theory of the deformed root systems |5| (see also formula (|35() 
below). The coordinates pi, qj can be considered as a version of the Frobenius 
coordinates on the partitions, so we will call them modified Frobenius (n, m)- 
coordinates (cf. jSlEI)- 

The image F{Hn,m) is dense in C""*""^ with respect to Zariski topology. 
The homomorphism 



C[xi, 



) ^^n, yi, ■ ■ ■ , Un 



is defined by the relation 

v\f){p,q) = f{{F-Hp,q)), 

where {p,q) € F{Hn,m)- In other words we consider the shifted symmetric 
function / as a function on the partitions from the fat hook and re-write 
it in the modified Frobenius (n, m)-coordinates. The fact that as a result 
we will have a polynomial is not obvious. To prove it consider the following 
shifted versions of the Bernoulli sums ©: 



^1(-,^) = E 



i>l 



Bk{z^ + \ + e{\-i))-Bk{\ + eC--i)) 



(29) 



Lemma 4. The image (p{f) of a shifted symmetric function / € Ag is a 
polynomial. For the Bernoulli sums b\.{z,6) it can be given by the following 
explicit formula: 



n 

{b\{z,e))=Y: 

i=l 



Bk{x^ + \ + \{m- en)) - Bk{\ + ^(^ - i)) 



+(- 



\k~l 



E 



Bk{y^ + \- \{e-^m - n)) - i?fe(i + n + e-\\ - j)) 



(30) 



Proof. Assume that Zi = Xi, where A G H„ 



Then we have 



i=l 



Bk{h + \ + e{\-i))-Bk{\ + e{]^-i)) 



i>l 



Bkil^i + ^ + 9{^-n-i))- Bk{^ + 0(1 - n - i)) 



18 



The same calculations as in Lemma 1 show that the last sum is equal to 



This proves the formula (|3U() . Since the Bernoulli sums generate Ag this 
implies the first part of the Lemma as well. 

Theorem 3. IfO is not a negative rational number or zero, then the image 
of the homomorphism if^ coincides with the algebra ^ ^ and the kernel of 

(f^ is spanned by the shifted Jack polynomials P^{z,9) corresponding to the 
Young diagrams which are not contained in the fat {n,m)-hook. 

Proof. The first claim follows from Lemma 4 and Theorem 2 from 
To prove the statement about the kernel consider a shifted Jack polynomial 
P^{z,9) with A G Hn^m- Let ;U be a partition whose diagram is contained 
in the fat (n, m)-hook. Since this implies that the diagram of A is not a 
subset of the one of fi according to the Extra Vanishing Property of shifted 
Jack polynomials (see Section 3) we have PxifJ^, d) = 0. Thus we have shown 
that P^{z,6) with A G -ffn,m belong to the kernel of if. To show that they 
generate the kernel one should note that 

ip\p-^{z,e)) = ^{Px{z, e)){xi,. ..,xn, -eyi, -6y^) + ..., 

where dots mean the terms of degree less than jA|. From Theorem 2 it 
follows that ip\P^{z,9)) with A G Hn. are linearly independent. Theorem 
is proved. 

Corollary. For generic 9 the functions 

SP*,{x,y,9) = ip\P*,{z,e)) 

with A G Hn^m form a basis in A^ ^ 

We will call the polynomials SP^{x, y, 9) the shifted super- Jack polynomi- 
als. They have the following independent definition (cf. [HI E| and section 
3). 

Let A and ly be two arbitrary partitions from the fat (n, m)-hook and 
(p(A), q{X)) be the corresponding modified Frobenius (n, m)-coordinates 
then the shifted super-Jack polynomials SP* are uniquely determined by the 
condition degS'P* < and the following property: 

r H{X), X = v 

SP;(P(A),,(A),«) = | „^ |A|<H,A^. 

where H{\) is the same as above (see formula Q in section 3). 

We have also the following version of the Extra Vanishing Property for 
the shifted super- Jack polynomials: 

SP:{p{X),q{X),9)=0 (32) 



unless the diagram of is a subset of the diagram of A. 
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Consider now the algebra of differential operators in n + m variables with 
rational coefficients belonging to C[xi, . . . , x„, yi, . . . , ym, {xi — Xj)~^, {xi — 
yi)~^, {Vk — xi)""^]. We denote it as V{n,m). 

Theorem 4. For generic values of 9 there exists a unique monomorphism 
V' ■ -'^n m 9 ^ 2?(n, m) such that the following diagram is commutative 

Ae ^ V{e) 

[ tp^ [ res 

where x is the inverse Harish- Chandra homomorphism and res is the oper- 
ation of restriction on the generalised discriminant described in the previous 
section. 

f f 

Indeed let / be a shifted symmetric function from Ag, Cq and ^nme ~ 

res{Cl) be the corresponding quantum integrals of CMS and deformed CMS 
problems. We know that if Px{z, 6) is a Jack symmetric function then 

<,n,eV'(^'A(^,^)) = /(AMPA(^,^)). 

Therefore according to Theorem 2 £^ ^ g = if and only if /(A) = for any 
A with the diagram contained in the fat (n, m)-hook. Now from Theorem 3 
it follows that Ker{res o x) = Kerip^. 

Let us denote by V^n, m, 9) the image of homomorphism ip. We claim 
that it is generated by the following quantum integrals of the deformed CMS 
problem Cp introduced in jSj. 

It will be convenient for us to change the notation now by introducing two 
sets of indices: /q = {1,2,..., n} and Ii = {1,2,..., rfi} and put Xj = yj. 
Let p denote the (parity) function on / = Iq U Ii such that p{i) = 0,i G Iq 
and p{i) = l,i £ Ii. 

Now define by induction the differential operators d^'^\ i € / as follows: 
for p = 1 

9f ) = (-^)^«..^ 

and for p > 1 

The differential operators Cp are defined as the sum 

Cp = Y,{-9)-^'^'^d^\ (34) 

Theorem 5. The operators Cp for allp = 1,2,... map the algebra A^^^m 6 
into itself. 

The proof is based on the following two technical lemmas. Let us de- 
note by Ji I the ideal in the polynomial algebra C[xi, . . . , x^, xj, . . . , Xm] 
generated by xi — xj. 
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Lemma 5. The following operators 

^;a(i) + aa)_i(e + i)|±£i 
4) (Of - e-Hd^'^^y - (9« - 

map the ideal Ji i into itself. 
The proof is straightforward. 

Lemma 6. The following operators 

1) sjp^-aj^) 

2) {d^^^ -d^^^)d^\i^i-i 
5;a(i)a(p)_aa)5(p) 

4){d^^ — d^^^){d^'^'^ — 9^^d'^~^^) map the algebra h.n,m,e into the ideal 

Jif. If f G I^n,m,e then d^/^ f is a polynomial. 
Proof is by induction with the use of Lemma 5. 
To prove Theorem 5 consider any polynomial / G 

According to 

Lemma 6 Cp{f) is also a polynomial. To show that it belongs to A„ we 
have to check that (Sj^^ — d'^'') {Cp) (/) belongs to the ideal j. But 

so due to Lemma 6 this is true since all the summands belong to j. 
Theorem 5 is proved. 

Now according to ^ the operators Cp commute with each other and in 
particular with C2, which is the deformed CMS operator. By the standard 
arguments (see e.g. proof of Theorem 1 from [3) one can show that they 
must commute also with the operators C^^^g for any / £ ^ g. 

Since the operators C{^^g separate the super-Jack polynomials SPx{x, y, 9) 
with A G Hn^m these polynomials must be also the eigenfunctions for the 
operators Cp. For any A € -ffn.m define u\ = (Ai, . . . , A^, //'i, . . . , /^m); where 
H = (A„+i, An+2, . . . ). It follows from the results of |51 that 



CpSPx{x, y, e) = Zp{ux - p)SPx{x, y, 9) 



where 



p^ = 9{i-]^) + ]^{m-9n), = 0-i(i - - i(^-im + n), (35) 

z = 1, . . . , n, j = 1, . . . , m and 

Zp{X) = \l + ... + xl + {-9n^,l + .■■ + ^,vJ + ,,,^ 

where dots mean the terms of less lexicographic order. It was also proven in 
[31 that for generic 9 Zp generate the algebra J^^me- This means that one 
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can express any operator £^^g as a polynomial of the operators L^. Thus 
we have proved the following 

Theorem 6. For generic values of 9 the operators Cp for p = 1,2,... 

generate the algebra T>{n,m, 9). 

7. Filters and CMS-invariant ideals in A 

The previous results lead naturally to the following question: which ideals 
/ in A are invariant under the action of CMS-operator Lq and its quantum 
integrals ? To answer this question it is useful to introduce the following 
notion which was probably first used by Knop and Sahi but we will 
follow here the terminology of the recent paper by Regev |27j . 

Let V be the set of all partitions ( or Young diagrams). 

Definition. The subset <Z V \s called filter if it is closed under the 
inclusion. In other words is a filter if for any diagram A G and any ji 
such that A C ^ it follows that fj, £ Q. 

Theorem 7. For generic 9 (more precisely, for 9 not a nonpositive ra- 
tional) there is a bijection between the set of CMS-invariant ideals in A and 
the set of filters. 

Proof. CMS-invariance implies that the corresponding ideal / is a linear 
span of certain Jack polynomials Px = Px{9): 

I = Span{Px{9),X£ Qj) 

for some C V. Let us prove that fi/ is a filter. Take any A G fJ/ and 
A C /i. We should show that P^ € /. When — A is one box then this 
follows from Fieri formula [SI E] 

PxPi = ^ij,/x{e)Pu, 

V 

where the sum is taken over partitions v obtained by adding one box to A. 
If j is such that Vj = Aj ■ + 1 then ipu/xi^^) is given by 

/ 'y^ {ij-i-i)e + K-Xj){ij-i + i)9 + x,-Xj-i) 

Vu/xK ) ii _ ^ . _ _ ^ 

Since 9 is not a negative rational number or zero all coefficients ipu/xi^^) 
nonzero. Now CMS-invariance of I implies that all Pi, in this sum belong 
to I. By induction the same is true for all u, such that A C z^. 
Now let be a filter. Let us show that 

J = Span{Px{9),X€ n) 

is an ideal in A. It is enough to show that PxSr G J, where is elementary 
symmetric function. But this is a direct consequence of the Fieri formula 

Pxer = ^^l^,/xiO)P. 

V 
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where now the sum is taken over partitions u such that A C and — A is 
a vertical r-strip. The coefficients i^u/x{0) have the following form (see jHj, 
VI.6): 

n ^ 



where 



d + 6 

b\(s) = -7 , if s € A, and 1 otherwise 



and C^^/x (resp. Ru/\) denote the union of the columns (resp. rows ) of z/ 
that intersect v/\ and Cg are given by the formula © above. Note that our 
assumption on 9 implies that Cg + 1 ^ 0. Theorem 7 now follows. 

We will denote the ideal / = Span{Px{6), X € il) corresponding to filter 

n as in{e). 

Let n{X^^\ A(")) be the set of Young dia grams fi such that fi contains 
at least one of the Young diagrams a«,...,aw. It is easy to see that 
rj(A(^\ . . . , A*-"-*) is a filter. We will call such a filter finitely generated. 

Lemma 7 (A. Regev |27j ) . Any filter is finitely generated. 

This result was proven by Regev using combinatorial arguments. We 
will give a simpler algebraic proof using our results from [3]. 

Proof of Lemma 7. Any ffiter contains some rectangle vr. Consider the 
ideal lTr{9) = Span{Px{9),TT C A) assuming that 6 is generic. It is obviously 
contained in /o(0). According to ^ for generic 9 the algebra A/I-^{9) is 
finitely generated (see Theorem 5 in 

Therefore the ideal In{9) / I-k{9) C h/lT^{9) is finitely generated as well 
(as an ideal in Noetherian algebra). Let Pxm {9), ■ ■ ■ , Px(") (^) Jack poly- 
nomials whose images generate I^{9) / It^{9) C A//^(0) . We claim that 
= r2(7r,A(^\...,A(")). Let A G then Pa G In- By assumption there 
exist /i ,...,/„ G A such that 

n 

Px-J2fiPxii) ^In 
1=1 

Now using again Fieri formula we conclude that Pa is a linear combination 
of Pi, where z/ contains either A^*-* or vr. Lemma is proved. 

Let now be a filter and consider the corresponding algebra Aq{9) = 
A/In{9). 

Theorem 8. For any filter Q and generic 9 the algebra Aq{9) is finitely 
generated. 

Proof. Consider a rectangle vr G and the corresponding ideal 

U{9) = Span{Px{9),XD tt) c In{9) 

Since A^(0) = A//^(0) for generic 9 is finitely generated the algebra Afi{9) 
is also finitely generated as a homomorphic image of A-,^{9). 

Consider now the special case when = Q{X) is generated by one Young 
diagram A. If A is (n + 1) x (m + 1) rectangle then for generic 9 according to 
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[Sj A;^(^) = A„^m,e- Note that when m = we have the standard symmetric 
polynomial algebra A„. The question is what happens for non-rectangular 
Young diagrams. 

Recall that an algebra is called a domain if it has no zero divisors , i. e. 
06 = implies that either o = or 6 = 0. An algebra is called nilpotent free 
if a*^ = implies o = 0. 

Theorem 9. For generic 9 the algebra AQ^x)iS) nilpotent free. It is 
a domain if and only if X is a rectangle. The ideal ln(x){0) is the inter- 
section of the prime ideals I.,^-{6), corresponding to the maximal rectangular 
subdiagrams vri, . . . , vr^ C A. 

Proof. First of all if A is a rectangle vr this follows from the results of [3] 
and Theorem 2 above since the corresponding algebra A-,^{6) can be realised 
as a subalgebra An,m,e in the polynomial algebra. Let now A be arbitrary. 
Consider all maximal rectangular subdiagrams tti, . . . , VTfc C A, then 

k 
i=l 

where each of the ideals I-wiiO) is prime. In particular, P-kSO) ^ In{x){^) 
but the product Pt^^{6) . . . P.,^^^{6) € ln{x){0)- Thus Aq(^x){(^) contains zero 
divisors unless A is rectangular. 

Show now that it is nilpotent free. Take any a £ A then if a" € In{X)i(^) 
then a"" € I-Ki{0) for alH = 1, . . . , fc. But since I-Ki{0) are prime this implies 
that a G Iwiid) and hence a G In(x)i(^)j which completes the proof. 

Corollary. Affine algebraic variety 

Mx{e) = SpecAn(^x){9) 

corresponding to a non-rectangular Young diangram A is reducible. Its ir- 
reducible components are the generalised discriminants related to maximal 
rectangular subdiagrams of X. 

Let us discuss now the filters generated by two Young diagrams Ai and 
A2. We restrict ourselves by the case when Ai = vri, A2 = vr2 are rectangular. 

Let vr = TTi n 7r2 be the intersection of these rectangles. We will assume 
that vr is different from vri and ^2- 

Theorem 10. //vri and 1^2 are two rectangular Young diagrams such that 
neither of them contains another then for generic 9 the algebra A^(^^^^^^-^{9) 
has nilpotent elements. 

Proof. Let the intersection vr of vri fl ^^2 be of size n x m. Consider the 
following polynomial 

P"n,m — Uj) 

It is easy to see that Rn,m £ An-m,e for any 9. One can represent it as a sum 

A 
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where A contains the n x m rectangle. Indeed, under the homomorphism 
sending Xn and i/n to zero Rn,m obviously becomes zero. This means that 
the corresponding Px{0) belong to the kernel of 

y^n-l,m-l '■ A > A„„i.m-1 

which is known to consist of Px{0) with A containing nx m rectangle. Con- 
sider now the smallest rectangle vr* which contains both vri and tt2. If tti 
and TT2 are n x M and N x m rectangles respectively, then vr* has the size 
iV X M. We claim that 

Rn,m^i---^n = ^ cxSPx{x,y,9) (36) 

as elements of Ajv,A^,e- Indeed 

Rn,m+i= ^ cxSPx{x,y,6) 

in A^^M+i.e- Putting yM+i = we came to (|36() . Now take the natural 
homomorphism 

Since the right hand side of (|36|) becomes zero we come to the relation 

Rn,m^i . . . Xat = 
in Af^(jri,7r2)(^)- Similarly one has the relation 

RnmVi ■ ■ ■ VM = 

in Aq(^j We claim that this implies that R% ^ = 0. Indeed Rn,m 

is the sum of monomials each of them is divisible either by the product 
xi . . . Xn OT yi . . . yM- Theorem 10 is proved. 

Conjecture. For generic 9 the radical of the ideal Iq^i^^^ .,^^-^{9) is equal to 
In(7T){^), where vr = tti n 7r2. 

Recall that the radical R{I) of the ideal / C A consists of the elements 
a € A such that a" G A for some n. It might help to prove this conjecture if 
the following Stanley conjecture about Jack polynomials would be true: if 

Pxie)p^{e) = Y,c'i^io)p,io) 

V 

is the expansion of the product of two Jack polynomials then the coefficient 
c'^^{9) / if and only if c^^(l) 7^ (see Conjecture 8.4 in 0). Unfortunately 
as far as we know it is still an open question. 

Thus we see that rectangular Young diag rams (and related algebras A^^^^^^^) 
play a very special role in the theory of filters and corresponding algebras 
A^{9). This shows a fundamental importance of the generalised discrimi- 
nants and gives another justification for our investigation. 
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8. Combinatorial formulas 



In this section we give some combinatorial formulas for the super-Jack 
polynomials and shifted super-Jack polynomials generalising the results by 
Stanley, Okounkov and Olshanski (see Let us recall these results. 

A tableau T on A is called a reverse tableau if its entries strictly decrease 
down the columns and weakly decrease in the rows. By T(n) denote the 
entry in the square □ € A. The following combinatorial formula for shifted 
Jack polynomial was proved by Okounkov in [12] : 

Pl{x,e) = Y,MO) n (^nn) - (37) 

where for a square □ = 

ce{a) = (j - 1) - e{i - 1) (38) 

(see formula (2.4) in JHI)- Here the sum is taken over all reverse tableaux 
on A with entries in {1, 2, . . . } and (Pt{G) is the same weight of tableau as 
in the combinatorial formula for ordinary Jack polynomials [ZI, IS]: 

Px{x,9) = Y,M0)llxTia)- (39) 

T Oex 

We should mention that in [S] sum in formula ()39() is taken over ordinary 
tableaux but since P\{x, 6) is symmetric it also holds if the sum in the right- 
hand side is taken over all reverse tableaux. We have also the following 
generalisation of H39|) for the skew Jack polynomials 

T DeX/fi 

where the sum is taken over all reverse tableaux of shape X/fi with entries 
in {1,...,} (see 0). 

Let us consider now a reverse bitableau T of type (n, m) and shape A . 
We can view T as a filling of a Young diagram A by symbols 1 < 2 • • • < n < 
l' < 2' • • • < m' such that its entries weakly decrease down the columns and 
right the rows, besides entries 1, 2 . . . , n strictly decrease down the columns 
and entries 1', 2' . . . , m' strictly decrease in rows. Let Ti be a subtableau in 
T containing all symbols 1' ,2' . . . ,m' and Tq = T — Ti. 

Theorem 11. For generic values of the parameter 9 the super- Jack poly- 
nomials can be written as 

SPx{xi,X2, ■ . .,Xn,yi,y2, ....ym) = ^^t{0) W Xt(U) (41) 

where Xji is denoted as yj and 



Proof follows directly from the formulas (j25]) . (j39|l . (|in|) . 
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To formulate a similar result for shifted super-Jack polynomials it will 
be convenient for us to use instead of the algebra ^ n the followine; alge- 
bra ^ g consisting of the polynomials p{xi, . . . , x„, yi, . . . , y^) which are 

symmetric in Xi — 9{i — = 1, . . . , n and yj — 6~^{j — 1), J = 1, . . . , m 
separately and satisfy the conditions 

f{xi + l,yj - 1) = f{xi,yj) 

on each hyperplane Xi -\- 9(1 — i) = 9{yj -|-n — 1)-|-1— j = for i = 1, . . . , n 
and j = 1, . . . , m. It is easy to check that the shift r 

r(xi) =xi+ Pi, T{yj) = yj + p^+j, 

p is given by the formula (|35|) . establishes an isomorphism between A^^g 
and ^ o 

Consider the homomorphism (p^ = T~^(p^ 

Recall that Hn^m is the set of partitions A such that A„,-|_i < m and 
PL = (Art+i, A„+2, • • • )• Consider the following map 

m ' ^ 

: F^[\) = {ai, . . . ,an,bi, . . . ,bm) where = Xi,i = l,...,n and bj = 
p'j,j = l,...,m . The set F^{Hn,m) is dense in C'^+"^ with respect to 
Zariski topology. It is easy to see that 

^\f){a,b) = fiF'-\a,b)) 

where (a, 6) G F^{Hn,m) and / G Ag. 

We are going to present a combinatorial formula for the following version 
of the shifted super-Jack polynomial 

SPi{x,y,9) = ^\Pl{z,9)). 

Recall that a reverse tableau T type (n, m) and shape A is a filling by symbols 
l<2<---<n<l'<2'<---<m' such that 

1) in each row (resp. column) of T the symbols decrease in the weak sense 
from left to right (resp. from top to bottom) 

2) there is at most one marked symbol j in each row and at most one 
unmarked symbol i in each column. 

By T(n) denote the entry in the square □ G A. 
Theorem 12. The following formula is true: 

SPi = 5^ MO) n i(-Of^^°^^XT^n) - cem, (42) 
T De\ 

where eg are given by hSf^). 

Proof. Let us consider the skew diagram \/p and define skew shifted 
Jack polynomials by the following natural generalisation of the Okounkov's 
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formula |^ 



In Okounkov proved that 

P*^izuZ2...,e) = Y,^x/f^i(^) n izi-cemp;{z2,zs...,9), (43) 

where fi ^ X means Aj+i < Hi < Xi and ^\/^{P) is the same coefficient as in 
the formula for the ordinary Jack polynomials 

At-<A 

(see [Z] and [01, section VI. 10). Applying the formula (|43j) n times we get 

P\{Z1,Z2 ...,0) = ^Pli^{zi,Z2, . . . ,Zn,9)Px{Zn+l,Zn+2 ■■■,0) 
AtCA 

and thus 

ip\Px{zi,Z2 . . . ,6*)) = ^ P^/^{xi,X2, . . . ,Xn,9)iJ*{P*{Zn+l,Zn+2 ■ ■ -,0)). 
AiCA 

Now using the duality (0) we have 

AtCA 



Pl{zi,Z2 ...,e)) = Y^ Pl/^{xi,X2, ...,Xn, ^) (^1' ^2 • • • , Vm, 1/0). 

But according to the formula (|37() 

P;,{yi,y2...,ym,l/e) = Y,VTia/0) H (^T^(n') " ci/e(n') 



J](^T,(1/^) n (^T,(n) + Iceia)] = (-1/0)1^1 ^(^Ti(l/^) n ((-^)^Ti(n) - ce{n)) , 

where T( is the reverse tableau conjugate to Ti. Therefore ip^{Pl{zi, Z2 ■ ■ ■ ,9)) 
can be rewritten as 

E(-v^)'"' f^r'/L '/^To(g) n K(n)-c.(n))^T,(i/0)n(M)^T,(n)-c.(n)) 

T DgA 

This completes the proof. 
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9. Some open questions. 



It seems that for the generic values of the parameter 6 the general situation 
is more or less clear now, so the main problems remain for the special values 
of the parameter 9. Let us mention some of them. 

For a given negative rational 6 is it true that the kernel of the homo- 
morphism if is generated by some Jack polynomials ? If yes, what is the 
geometry /combinatorics of the corresponding Young diagrams ? The best 
results known so far in this direction can be extracted from the paper 
(see also P3|). 

How to describe the generators of the algebra A„ „^ g for special values of 
6 ? What are the corresponding Poincare series ? For generic 9 the answer 
to the last question was given in jKj. 

How is this related to the theory of quasi-invariants for the deformed root 
systems j26| ? In particular, are there interesting extensions of the algebra 
A„ for special values of ^ ? We know that at least for m = I the answer 
is positive (see [2^1 for details). 

There are also several important open questions left in the case of generic 
9. In particular, as we have already mentioned above (see Remark 1 in 
section 5) the finding of a convenient set of generators for the algebra A„ ,„,6» 
and a satisfactory description of the relations between them is still to be 
done. Another problem is to extend our investigation of the algebras Aq{9) 
to a general filter fi. 

A natural question also is about the generalisations of our results for 
the deformed Macdonald-Ruijsenaars operators introduced in 0. We are 
planning to discuss this in a separate paper. 
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